Reliable Classifiers in ROC Space
نویسندگان
چکیده
The performance of a classifier can be improved by abstaining on uncertain instance classifications. The transformation from the original Receiver Operator Characteristic (ROC) curve to the curve obtained by abstention is provided. We include proofs on dominance of this new ROC curve to aid classifier selection and to show the effectiveness of the approach. For specific cost and class distributions we provide an approach in ROC space to transform a classifier into a new one that has a desired precision per class.
منابع مشابه
An Analysis of Reliable Classifiers through ROC Isometrics
Reliable classifiers abstain from uncertain instance classifications. In this paper we extend our previous approach to construct reliable classifiers which is based on isometrics in Receiver Operator Characteristic (ROC) space. We analyze the conditions to obtain a reliable classifier with higher performance than previously possible. Our results show that the approach is generally applicable to...
متن کاملThe ROC isometrics approach to construct reliable classifiers
We address the problem of applying machine-learning classifiers in domains where incorrect classifications have severe consequences. In these domains we propose to apply classifiers only when their performance can be defined by the domain expert prior to classification. The classifiers so obtained are called reliable classifiers. In the article we present three main contributions. First, we est...
متن کاملIterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs
Hidden Markov models (HMMs) have been shown to provide a high level performance for detecting anomalies in sequences of system calls to the operating system kernel. Using Boolean conjunction and disjunction functions to combine the responses of multiple HMMs in the ROC space may significantly improve performance over a ‘‘single best’’ HMM. However, these techniques assume that the classifiers a...
متن کاملCautious Classifiers
The evaluation and use of classifiers is based on the idea that a classifier is defined as a complete function from instances to classes. Even when probabilistic classifiers are used, these are ultimately converted into categorical classifiers that must choose one class (with more or less confidence) from a set of classes. Evaluation metrics such as accuracy/error, global cost, precision, recal...
متن کاملPosition Paper: Extensions of ROC Analysis to Multi-Class Domains
Receiver-operating characteristic (ROC) analysis has proven to be a powerful method for dealing with misclassification costs and skewed class distributions (Provost & Fawcett, 1998). In the typical representation, an ROC analysis evaluates false accept versus false reject rates for a set of candidate binary classifiers under all possible (linear) cost and prior class distribution assumptions. T...
متن کامل